One-step dejittering of digital video images

We propose several fast (quasi real-time) methods to dejitter digital video images in one step. They are based on an essential disproportion of the magnitude of the second-order vertical differences along the columns of a real-world image and all its jittered versions. These methods need one iteration to dejitter the image, which involves a number of steps equal to the number of its rows. They are designed for gray-value and color images (when the jitter is the same for all color channels), as well as to slightly noisy images. A reasonable version of these methods can be considered as parameter-free. We propose a two-phase extension enabling to restore jittered images corrupted with strong white noise (10 db snr or less). We consider specific error measures adapted to assess the success of dejittering. The results we obtain outperforms by far the existing methods: first in quality, second in speed (the ours need no more than 1 second for a 512 × 512 image on Matlab). Our methods provide a crucial step towards real-time dejittering of digital video sequences. 1 Intrinsic dejittering Image jittering consists in a random horizontal displacement of each line of the image. Jittering occurs e.g. in video images when the synchronization signals—carrying the information about the proper location of the lines relative to each other—are corrupted by noise or degradation of the storage medium. Line jittering can also appear in wireless video transmission, due to some electromagnetic interference. The visual effect is quite disturbing since each line is randomly displaced within a range ±1 to ±5 pixels or more. Then shapes appear to be jagged in the vertical direction—a first example is shown in Fig. 1 (c). Another form—structured (e.g. sinusoidal jitter)—can be provoked by acoustic, electrical or other interferences [10]. The rows of the image are displaced with a number of pixels corresponding to the frequency and the amplitude of the electrical perturbation; then vertical lines are transformed into sinusoids—an example can be seen in Fig. 10. Time base corrector machines process with some success the analogue video signal in order to recover the line synchronization information [9]. In many cases, such an operation is unsuccessful or impossible. The alternative approach—to restore the image frames directly from the observed jittered data—is often called intrinsic dejittering [10]. It naturally uses some prior assumptions on natural images. We focus on such an approach since it is more flexible and more widely applicable. The very trivial solution to this problem—searching for the shift between consecutive lines that maximizes their correlation—is known to cause a bias towards vertical lines and fails in most of the cases [9]. Intrinsic dejittering was really inaugurated by Kokaram et al. in [8]. An more mature version of the method is nicely explained in chapter 5 of the textbook on film and video of Kokaram [9]. The algorithm presented there is based on a 2-D autoregressive model (2D AR) of the image. The unknown 2D AR coefficients and the erroneous horizontal displacements are considered by blocks. The are estimated jointly using an iterative algorithm. Drift compensation may be necessary to finalize the restoration. Laborelli proposes in [10] a totally different approach where the l1 norm of the differences between consecutive shifted (two or three) lines is compared. The optimal shifts are recovered during a backward iteration in the context of a dynamic programming approach. Later on, Shen proposed a fully Bayesian method using a TV model on the underlying image [15] for joint dejittering and denoising. It requires to minimize a non-smooth function for each frame, which is time consuming. A more